Papers with symbolic reasoning
Machine Reasoning: Technology, Dilemma and Future (2020.emnlp-tutorials)
Copied to clipboard
| Challenge: | Machine reasoning is a field of research that aims to build interpretable AI systems . Symbolic reasoning methods represent knowledge using symbolic logic, perform inference . probabilistic reasoning methods combine probability and symbolic logic into a unified model . |
| Approach: | This tutorial introduces machine reasoning frameworks and aims to define them . it will show how they can be used to build interpretable AI systems . |
| Outcome: | This tutorial aims to show how machine reasoning frameworks perform in real-world scenarios. |
Does Self-Consistency Improve the Recall of Encyclopedic Knowledge? (2026.acl-short)
Copied to clipboard
| Challenge: | a lack of evaluation grounds for self-consistency on symbolic reasoning is unclear . however, it is unclear whether it improves performance on non-math questions involving encyclopedic knowledge. |
| Approach: | They establish a knowledge recall split for the popular MMLU benchmark by applying a data-driven heuristic from prior work. |
| Outcome: | The proposed knowledge recall split achieves an 89% accuracy on the MMLU benchmark. |
SymbolicThought: Integrating Language Models and Symbolic Reasoning for Consistent and Interpretable Human Relationship Understanding (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing methods for constructing character relationships from plain text are time-consuming and low in coverage. |
| Approach: | They propose a human-in-the-loop framework that combines LLM-based extraction with symbolic reasoning. |
| Outcome: | The proposed framework improves annotation accuracy and consistency while significantly reducing time cost. |
Rethinking Tabular Data Understanding with Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are capable of various tasks, yet their capability in interpreting and reasoning over tabular data remains an underexplored area. |
| Approach: | They propose a method for table structure normalization to improve model performance . they propose aggregation of multiple reasoning pathways to improve performance based on textual and symbolic reasoning. |
| Outcome: | The proposed method improves performance on symbolic reasoning tasks with textual reasoning slightly outperforming symbolic reasoning on tables. |
SpatialMath: Spatial Comprehension-Infused Symbolic Reasoning for Mathematical Problem-Solving (2026.findings-eacl)
Copied to clipboard
| Challenge: | Current models struggle to accurately decompose intricate visual inputs and connect perception with structured reasoning, leading to suboptimal performance. |
| Approach: | They propose a Spatial Comprehension-Infused Symbolic Reasoning Framework to integrate spatial representations into structured symbolic reasoning chains. |
| Outcome: | The proposed framework outperforms existing models in vision-intensive mathematical problems. |
Frequency Effects on Syntactic Rule Learning in Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract symbols and rules. |
| Approach: | They investigate the performance of BERT on English subject–verb agreement by analyzing word frequency and absolute frequency of verb forms. |
| Outcome: | The proposed model generalizes well to subject–verb pairs that never occurred in training, suggesting a degree of rule-governed behavior. |
HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation (2022.acl-long)
Copied to clipboard
Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, Dongmei Zhang
| Challenge: | Existing studies on table reasoning focus on flat tables and hierarchical tables . a new dataset, HiTab, aims to examine numerical reasoning over hierarchic tables based on hierarchically structured tables - a strong challenge for existing baselines and a valuable benchmark for future research. |
| Approach: | They propose a hierarchical question answering and natural language generation dataset to study hierarchic tables. |
| Outcome: | The proposed model shows that it is effective in QA and natural language generation over hierarchical tables. |
Empirical Investigation of Neural Symbolic Reasoning Strategies (2023.findings-eacl)
Copied to clipboard
Yoichi Aoki, Keito Kudo, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Neural reasoning accuracy improves when generating intermediate reasoning steps. |
| Approach: | They decompose the reasoning strategy w.r.t. step granularity and chaining strategy. |
| Outcome: | The proposed reasoning strategy significantly affects performance in a symbolic reasoning dataset. |
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks. |
| Approach: | They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity. |
| Outcome: | The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions. |
Table Fact Verification with Structure-Aware Transformer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models cannot be used to encode semi-structured data because of their nature. |
| Approach: | They propose a Structure-Aware Transformer which injects table structural information into mask . method could combine symbolic and linguistic reasoning, they propose . |
| Outcome: | The proposed method outperforms baseline on a large scale table verification dataset. |
PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models excel in various tasks, but their huge size and inaccessibility of parameters present challenges for practical deployment. |
| Approach: | They propose to use CoT data to distill task-specific ability from large language models to smaller models . they use reasoning programs to suppress errors in distilled data and improve distillation quality . |
| Outcome: | The proposed model outperforms LLMs on arithmetic reasoning, symbolic reasoning, and general ability. |
NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in parsing textual data and generating code. |
| Approach: | They propose a framework that normalizes web tables to support symbolic reasoning . they compare NormTab to a stand-alone framework that can normalize tabular data . |
| Outcome: | The proposed framework outperforms traditional models on symbolic reasoning tasks on tabular data. |
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021.acl-long)
Copied to clipboard
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua
| Challenge: | Existing QA systems focus on unstructured text, structured knowledge base, or semi-structured tables. |
| Approach: | They propose a large-scale question answering model based on financial reports . numerical reasoning is usually required to infer the answer . |
| Outcome: | The proposed model achieves 58.0% inF1, an 11.1% increase over the baseline model, but still lags behind the best human model. |
Adaption-of-Thought: Learning Question Difficulty Improves Large Language Models for Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods do not differentiate question difficulty when designing prompting methods for them. |
| Approach: | They propose an adaptive method to improve large language models for reasoning problems by measuring question difficulty and tailoring demonstration set construction and difficulty-adapted retrieval strategies. |
| Outcome: | The proposed method shows an absolute improvement of up to 5.5% on arithmetic reasoning, 7.4% on symbolic reasoning, and 2.3% on commonsense reasoning. |
UTMath: A Benchmark for Math Evaluation with Unit Test (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prevailing benchmarks for mathematical reasoning include MATH and AIME . predicated on single-instantiation problems with fixed numbers, these models leave generalization on isomorphic problem variants untested. |
| Approach: | They propose a mathematical reasoning benchmark that quantifies solution accuracy and solution space generality. |
| Outcome: | The proposed model solves 1,053 problems spanning 9 mathematical domains . the best-performing model solved only 32.57% of the problems . |
Query4Regex: Verifiable Regex Transformation through Formal Operations from NL and DSL Queries (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate structured data, but their ability to precisely manipulate it remains relatively under-explored. |
| Approach: | They propose a benchmark to evaluate verifiable transformations on regexes . they use natural language instructions and a program-like domain-specific language that specifies the sequence of operations to evaluate LLMs. |
| Outcome: | The proposed benchmark compares LLM performance on natural language and DSL queries for regex manipulation. |
Interactive Symbol Grounding with Complex Referential Expressions (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on symbol grounding models (grounders) uses lazy few-shot learning to relate open-class words like green and above to their visual percepts; and symbolic reasoning with closed-class word categories like quantifiers and negation. |
| Approach: | They propose a procedure for learning to ground symbols from a sequence of stimuli consisting of an arbitrarily complex noun phrase and its designation in the visual scene. |
| Outcome: | The proposed procedure is based on a visual reference resolution task in which the learner is unaware of concepts that are part of the domain model and how they relate to visual percepts. |
Reasoning in Large Language Models Through Symbolic Math Word Problems (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized NLP by solving downstream tasks with little to no labeled data. |
| Approach: | They propose a model that uses symbolic expressions to provide a concise explanation of the numeric answer. |
| Outcome: | The proposed model has good accuracy on symbolic word problems and is able to provide a concise and verifiable reasoning and make it more interpretable. |
Exploring Backward Reasoning in Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Multi-step reasoning through in-context learning strategies have been extensively explored, highlighting the abilities of Large Language Models (LLMs) to solve problems in a step-wise manner. |
| Approach: | They propose to use Large Language Models to generate answers from step-by-step reasoning by re-constructing the original question that led to the final answer. |
| Outcome: | The proposed models show that they are able to reason about the conclusion and reconstruct the original question that led to the final answer. |
H-STAR: LLM-driven Hybrid SQL-Text Adaptive Reasoning on Tables (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for tabular reasoning combine textual and symbolic reasoning in a two-stage process to address these limitations. |
| Approach: | They propose an algorithm that integrates symbolic and semantic (textual) approaches in a two-stage process to address these limitations. |
| Outcome: | The proposed algorithm outperforms state-of-the-art methods across three tabular question-answering and fact-verification datasets, underscoring its effectiveness and efficiency. |
MultiHiertt: Numerical Reasoning over Multi Hierarchical Tabular and Textual Data (2022.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for numerical reasoning over hybrid data only include a single flat table in each document . |
| Approach: | They propose a new benchmark with QA pairs over multi hierarchical tabular and textual data. |
| Outcome: | The proposed model is more complex and challenging than existing benchmarks and is available on github . it uses facts retrieving to extract relevant facts from both tables and text and symbolic reasoning over retrieved facts. |
Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks (2021.acl-long)
Copied to clipboard
| Challenge: | Existing solutions for math word problems lack explicit integration of math symbolic constraints, leading to unexplainable and unreasonable predictions. |
| Approach: | They propose a novel mathematical model that explicitly incorporates symbolic constraints by auxiliary tasks to enforce different symbolic reasoning. |
| Outcome: | The proposed solver incorporates symbolic constraints by auxiliary tasks to enforce different symbolic reasoning. |
Learn to Combine Linguistic and Symbolic Information for Table-based Fact Verification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification lack attention to combine linguistic and symbolic information. |
| Approach: | They propose a graph-based reasoning approach that learns to combine linguistic and symbolic information effectively. |
| Outcome: | The proposed method can combine linguistic and symbolic information effectively. |
Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for solving geometric problems are either small in scale or not publicly available. |
| Approach: | They propose a large-scale benchmark for geometric problem solving using formal language and symbolic reasoning. |
| Outcome: | The proposed approach parses geometry problems into formal language and performs symbolic reasoning step by step. |
Can Vision-Language Models Solve Visual Math Equations? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Vision-Language Models (VLMs) perform well on textual equations, but fail on visually grounded counterparts. |
| Approach: | They propose to decompose visual equation solving into symbolic equation solving and visual recognition into two core components to understand this gap. |
| Outcome: | The proposed models perform well on textual equations, but fail on visual grounded ones. |
PORT: Preference Optimization on Reasoning Traces (2025.naacl-long)
Copied to clipboard
| Challenge: | Preference optimization methods have been successfully applied to improve the alignment of large language models with human values. |
| Approach: | They propose to use preference optimization methods to generate rejected answers using weak LLM prompting and digit corruption to improve the mathematical reasoning abilities of language models. |
| Outcome: | The proposed method leads to increased accuracy on the GSM8K and AQuA-RAT benchmarks without annotations. |
Express What You See: Can Multimodal LLMs Decode Visual Ciphers with Intuitive Semiosis Comprehension? (2025.findings-acl)
Copied to clipboard
| Challenge: | Traditional VQA benchmarks encounter a modality gap and over-reliance on language priors, whereas human cognition excels at intuitive semiosis, associating abstract visual symbols to linguistic semantics. |
| Approach: | They propose a task of generating abstract linguistics from emoji sequence images, where such reasoning underpins critical applications in cryptography. |
| Outcome: | The proposed model can generate abstract linguistics from emoji sequence images, challenging MLLMs’ reasoning of decoding complex semantics of visual ciphers. |
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a new approach to prompt large language models (LLMs) but most studies rely on human-annotated rational chains to prompt LLMs . |
| Approach: | They propose a method that augments rational chains from a small labeled dataset and pruning low-quality chains to construct a pool of machine generated rationale chains based on the labels. |
| Outcome: | The proposed method can bypass human engineering of CoT by automatically augmenting rational chains from a small labeled dataset, and pruning low-quality chains to construct a candidate pool of machine generated rationale chains based on the labels. |
DecisionFlow: Advancing Large Language Model as Principled Decision Maker (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current language models lack the structured deliberation needed for high-stakes tasks such as healthcare and finance. |
| Approach: | They propose a decision-making framework that guides models to reason over structured representations of actions, attributes, and constraints. |
| Outcome: | The proposed framework achieves up to 30% accuracy gains over strong prompting baselines and enhances alignment in outcomes. |
Puzzled by Puzzles: When Vision-Language Models Can’t Take a Hint (2025.emnlp-main)
Copied to clipboard
| Challenge: | rebus puzzles encode language through imagery, spatial arrangement, and symbolic substitution. |
| Approach: | They construct a benchmark of rebus puzzles in english language to test their ability to interpret and solve them. |
| Outcome: | The proposed model performs well on a set of english-language rebus puzzles. |
Whiteboard-of-Thought: Thinking Step-by-Step Across Modalities (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have shown promising results in arithmetic and symbolic reasoning by expressing intermediate reasoning in text as a chain of thought, yet struggle to extend this capability to answer text queries that are easily solved by visual reasoning. |
| Approach: | They propose a method to unlock the visual reasoning capabilities of multimodal large language models by using a metaphorical ‘whiteboard’ to draw out reasoning steps as images and return these images back to the model for further processing. |
| Outcome: | The proposed method shows that it can be used on four difficult tasks that involve visual and spatial reasoning with no demonstrations or specialized modules. |
DRAE: Dynamic Retrieval-Augmented Expert Networks for Lifelong Learning and Task Adaptation in Robotics (2025.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that Dynamic Retrieval-Augmented Expert Networks outperforms baseline approaches in long-term task retention and knowledge reuse. |
| Approach: | They propose a dynamic routing architecture that leverages MoE and Retrieval-Augmented Generation to augment the learning process. |
| Outcome: | The proposed architecture outperforms baseline approaches in long-term task retention and knowledge reuse. |
Critic Rule Induction: Improving Temporal Knowledge Graph Forecasting with Generator-Critic Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for predicting future facts from time-evolving graphs rely on statistical co-occurrences and extensive path enumeration. |
| Approach: | They propose a Critic-Guided Rule Induction method which treats temporal rules as rule hypotheses to be examined and adopts a decoupled Generation-Discrimination pipeline to induce rules that are high-coverage and high-precision. |
| Outcome: | The proposed method outperforms strong baselines on three benchmarks and achieves state-of-the-art performance. |
Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series? (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Multimodal LLMs (MLLMs) show strong performance in complex reasoning tasks, but their ability to extract symbolic laws from time series data remains underexplored. |
| Approach: | They propose a benchmark to assess symbolic reasoning over real-world time series across three tasks: multivariate symbolic regression, Boolean network inference, and causal discovery. |
| Outcome: | The proposed framework integrates LLMs with genetic programming to form a closed-loop symbolic reasoning system. |